Fill out this form to register your intent to complete this project.
Fill out this form to submit your solution to this project and qualify for the rewards.
 | Design and train a deep learning model to compose music. |
Generative deep learning models have shown promise in several areas, particularly text and image generation. A musical piece can be described by a language, is often highly structured, and pleasing music lies in a low-dimensional subspace embedded within all possible combinations of notes and sounds. Thus, it is plausible that similar models could be applied to music composition without human supervision. This is an enticing prospect for, e.g., media production companies, as bespoke music could be generated for any purpose without relying on ‘stock music’ marketplaces.
Use the Deep Learning Toolbox™ and the Audio Toolbox™ to create a generative model that can compose and play back a melody. A melody can be represented using the MIDI format to describe a collection of musical notes, which come together to form a piece. One option is to train a generative model on collections of MIDI sequences to predict the next note(s) in a sequence, and this can then be used recurrently to compose entirely new melodies.
Suggested steps are:
-Perform a literature review and decide on a network architecture to use (e.g., an LSTM).
Decide on the style of music and download an appropriate training data set, e.g., a collection of classical music
Load and pre-process the training data as necessary.
Design and train a generative network on the training data set.
Play the resultant sequence, either by writing a MIDI file and playing it outside of MATLAB, or synthesizing audio with the Audio Toolbox™.
Assess the quality of the outputs from the network and make adjustments as necessary.
Advanced extensions:
Instead of a melody, try to generate chord sequences. This could then be combined with the melody to form a more sophisticated piece.
Instead of a single melody, try to generate a polyphonic piece, or a piece with multiple instruments. This could include percussion.
Make use of note properties other than pitch, such as velocity (volume). This can produce more expressive pieces.
Long Short-Term Memory Networks in MATLAB
Example Project Includes tools useful for reading and writing MIDI files.
Design and play a MIDI synthesizer
[1] Allen Huang and Raymond Wu, Deep Learning for Music
[2] Carlos Hernandez-Olivan and Jose R. Beltran, Music Composition with Deep Learning: A Review, arXiv:2108.12290
Generative music models can be used to create new assets on demand.
Artificial Intelligence, Deep Learning, Machine Learning, Neural Networks, Audio
Master's, Doctoral
Dedicated discussion forum to ask/answer questions, comment, or share your ideas for solutions for this project.
243
